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Abstract. Let F be either R or C. Consider the standard embedding GL n (F) «-» GL n+ i(_F) 
and the action of GL n (_F) on GL n +i(_F) by conjugation. 

In this paper we show that any GL n (F)-invariant distribution on GL n +i(_F) is invariant with 
respect to transposition. 

We show that this implies that for any irreducible admissible smooth Frechet representations 
7T of GL n +i (F) and t of GL„(F), 

dimHom GIj71 (F)(7r, r) < 1. 
For p-adic fields those results were proven in [AGRS . 
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1. Introduction 



Let F be an archimedean local field, i.e. F = K or F = C. Consider the standard imbedding 
GL n (F) <—* GL n+ i(F). We consider the action of GL ra (F) on GL ra+ i(F) by conjugation. In this 
paper we prove the following theorem: 

Theorem. A. Any GL n (F) - invariant distribution on GL n+ i(_F) is invariant with respect to 
transposition. 

It has the following corollary in representation theory. 

Theorem. B. Letix be an irreducible admissible smooth Frechet representation of GL n+ i(F) and 
t be an irreducible admissible smooth Frechet representation o/GL„(F). Then 

(1) dimHom GLri(F) (7r,-r) < 1. 

We deduce Theorem B from Theorem A using an argument due to Gelfand and Kazhdan adapted 
to the archimedean case in |AGS| . 

Property JT]) is sometimes called strong Gelfand property of the pair (GL n+ i(F), GL„(F)). It is 
equivalent to the fact that the pair (GL n+ i(F) x GL„(F), AGL„(F)) is a Gelfand pair. 

Remark 1.0.1. Using the tools developed here, combined with jAGRSj . one can easily show that 
Theorem A implies an analogous theorem for the unitary groups. 

Remark 1.0.2. After the completion of this work we found out that Chen-Bo Zhu and Sun Binyong 
have obtained the same results simultaneously, independently and in a different way, see [SZ . 
They also proved an analogous theorem for the orthogonal groups. 

1.1. Some related results. 

For non- archimedean local fields of characteristic zero theorems A and B were proven in [A"GRS . 
The current paper heavily uses the theory of D-modules, which cannot be directly applied to the 
non-archimedean case. For this reason, currently there is no uniform proof for all local fields. 
However, we hope that one could develop non-archimedean analogues of the D-module techniques 
that we use here. 

In [AGSj , a special case of Theorem B was proven for all local fields; namely the case when r is 
one-dimensional. 

Theorem A has the following corollary. 

Corollary. Let P n C GL„ be the subgroup consisting of all matrices whose last row is (0, ...,0, 1). 
Let GL„ act on itself by conjugation. Then every P n (F) - invariant distribution on Gh n (F) is 
GL„(i ? ) - invariant. 

This has been proven in [Barj for eigendistributions with respect to the center of E/c(gl n ). In 
[Bar| it is also shown that this implies Kirillov's conjecture. 
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1.2. Structure of the proof. 

We will now briefly sketch the main ingredients of our proof of Theorem A. 

First we show that we can switch to the following problem. The group Gh n (F) acts on a certain 
linear space X n and a is an involution of X n . We have to prove that every GL n (_F)-invariant 
distribution on X n is also er-invariant. We do that by induction on n. Using the Harish-Chandra 
descent method we show that the induction hypothesis implies that this holds for distributions on 
the complement to a certain small closed subset S C X n . We call this set the singular set. 

Next we assume the contrary: there exists a non-zero GL„(i 7 ')-invariant distribution (onl 
which is anti-invariant with respect to a. 

We use the notion of singular support of a distribution from the theory of D-modules. Let 
T C T* X denote the singular support of £. Using Fourier transform and the fact any such 
distribution is supported in S we obtain that T is contained in S where S is a certain small subset 
in T*X. 

Then we use a deep result from the theory of D- modules which states that the singular support 
of a distribution is a coisotropic variety in the cotangent bundle. This enables us to show, using a 
complicated but purely geometric argument, that the support of £ is contained in a much smaller 
subset of S. 

Finally it remains to prove that any GL„(i ;l )-invariant distribution that is supported on this 
subset together with its Fourier transform is zero. This is proven using Homogeneity Theorem 
(Theorem 12. 2. 12p which in turn uses Weil representation. 



1.3. Content of the paper. 

In section [5] we give the necessary preliminaries for the paper. 
In subsection 12.11 we fix the general notation that we will use. 

In subsection 12 . 21 we discuss invariant distributions and introduce some tools to work with them. 
The most advanced are 

• The Homogeneity theorem and Fourier transform. 

• The Harish-Chandra descent method. 

In subsection [213] we discuss the notion of singular support of a distribution. The most important 
for us property of this singular support is being coisotropic. This fact is a crucial tool of this paper. 
In subsection 12 .41 we introduce notation that we will use in our proof. 

In section [3] we use the Harish-Chandra descent method. 

In subsection 13 . 1 1 we linearize the problem to a problem on the linear space X — sl(V) x V x V* , 
where V = F n . 

In subsection 13 . 21 we perform the Harish-Chandra descent on the sl(V)-coordinate and V x V* 
coordinate separately and then use automorphisms v\ of X to descend further to the singular set S. 

In section 0] we reduce Theorem A to the following geometric statement: any coisotropic subva- 
riety of S is contained in a certain set Cxxx- The reduction is done using the fact that the singular 
support of a distribution has to be coisotropic, and the following proposition: any GL(U)-invariant 
distribution on X such that it and its Fourier transform are supported onsl(U) x (V X OUOx V*) 
is zero. 

In subsection 14.11 we prove this proposition using Homogeneity theorem. 

In section[S]we prove the geometric statement. This is the most complicated part of the paper. 
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In subsection 15.11 we give preliminaries on coisotropic subvarieties. In particular, we give a 
geometric partial analog of Frobenius reciprocity for coisotropic subvarieties ( Corollaries 15 . 1 . 7l and 
15X8)1 . 

In subsection !5.2l we stratify the set S and use an inductive argument on the strata. This reduces 
the geometric statement to a proposition on one stratum that we call the Key Proposition. 

In subsection 15.31 we analyze a stratum of S and then use the geometric analog of Frobenius 
reciprocity to reduce the Key Proposition to a lemma on V x V* x V x V* that we call the Key 
Lemma. 

In subsection 15 .41 we prove the Key Lemma. 

In Appendix [A] we prove that Theorem A implies Theorem B using an archimedean analog of 
Gelfand-Kazhdan technique. 

In Appendix [5] we give more details on the facts concerning the theory of D-modules listed in 
subsection 12.31 

1.4. Acknowledgements. 

We thank Joseph Bernstein for our mathematical education. We thank Joseph Bernstein, 
David Kazhdan, Bernhard Kroetz, Eitan Sayag and Gerard Schiffmann for fruitful dis- 
cussions. We also thank Moshe Baruch, Erez Lapid and Siddhartha Sahi for useful remarks. 

Part of the work on this paper was done while we visited the Max Planck Institute for Mathe- 
matics in Bonn. This visit was funded by the Bonn International Graduate School. 

2. Preliminaries 

2.1. General notation. 

• In this paper all the algebraic varieties are defined over F. 

• For an algebraic variety X we denote by X(F) the topological space or smooth manifold 
of F points of X. 

• We consider linear spaces as algebraic varieties and treat them in the same way. 

• For an algebraic variety X we denote by Xc its complexification X Xg poc R SpecC. Note 
that if X is defined over M then Xc = X x X. 

• For a group G acting on a set X and a point x £ X we denote by Gx or by G(x) the orbit 
of x and by G x the stabilizer of x. 

• An action of a Lie algebra g on a (smooth, algebraic, etc) manifold M is a Lie algebra 
homomorphism from g to the Lie algebra of vector fields on M. Note that an action of a 
(Lie, algebraic, etc) group on M defines an action of its Lie algebra on M. 

• For a Lie algebra g acting on M, an element a € g and a point x € M we denote by 
a(x) 6 T X M the value at point x of the vector field corresponding to a. We denote by 
qx C T X M or by g(x) the image of the map a i— » a(x) and by g x C 9 its kernel. 

• For manifolds L C M we denote by N^f :— (Tm\l)/Tl the normal bundle to L in M. 

• Denote by CN^ 1 := (N^ 1 )* the conormal bundle. 

• For a point ygLwe denote by Njf y the normal space to L in M at the point y and by 
CNff the conormal space. 

2.2. Invariant distributions. 

2.2.1. Distributions on smooth manifolds. 

Notation 2.2.1. Let X be a smooth manifold. Denote by C^°(A) the space of test functions on 
X, that is smooth compactly supported functions, with the standard topology, i.e. the topology of 
inductive limit of Frechet spaces. 
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Denote V{X) := C™(X)* to be the dual space to C C °°(X). 

For any vector bundle E over X we denote by {X, E) the space of smooth compactly supported 
sections of E and by T>(X, E) its dual space. Also, for any finite dimensional real vector space V 
we denote C™(X,V) := C™(X,X x V) and T>(X, V) := V(X,X x V), where X x V is a trivial 
bundle. 

2.2.2. Schwartz distributions on Nash manifolds. 

Our proof of Theorem A widely uses Fourier transform which cannot be applied to general dis- 
tributions. For this we require a theory of Schwartz functions and distributions as developed in 
|AG1| . This theory is developed for Nash manifolds. Nash manifolds are smooth semi-algebraic 
manifolds but in the present work only smooth real algebraic manifolds are considered. Therefore 
the reader can safely replace the word Nash by smooth real algebraic. 

Schwartz functions are functions that decay, together with all their derivatives, faster than any 
polynomial. On M. n it is the usual notion of Schwartz function. For precise definitions of those 
notions we refer the reader to [AGlj . We will use the following notations. 

Notation 2.2.2. Let X be a Nash manifold. Denote by S(X) the Frechet space of Schwartz 
functions on X . 

Denote by S*(X) := S(X)* the space of Schwartz distributions on X . 

For any Nash vector bundle E over X we denote by S(X, E) the space of Schwartz sections of 
E and by S* (X, E) its dual space. 

Notation 2.2.3. Let X be a smooth manifold and let Z C X be a closed subset. We denote 

S* X {Z) G S*PO|Supp(OcZ}. 

For a locally closed subset Y C X we denote S X (Y) :— 5jL y . (Y). In the same way, for any 

bundle E on X we define S x ( Y, E) . 

Remark 2.2.4. Schwartz distributions have the following two advantages over general distribu- 
tions: 

(i) For a Nash manifold X and an open Nash submanifold U C X, we have the following exact 
seguence 

0^S* x (X\U)^ S*(X) -> S*(U) -» 0. 

(ii) Fourier transform defines an isomorphism T : iS*(]R n ) — > 5*(M n ). 

The following theorem allows us to switch between general distributions and Schwartz distribu- 
tions. 

Theorem 2.2.5. Let a reductive group G act on a smooth affine variety X. Let V be a finite 
dimensional continuous representation of G(F) over R. Suppose thatS*(X(F),V) G ^ = 0. Then 
V{X{F),V) G ^ = 0. 

For proof see [AG2j . Theorem 4.0.8. 

2.2.3. Basic tools. 

We present here some basic tools on equivariant distributions that we will use in this paper. 

Proposition 2.2.6. Let a Nash group G act on a Nash manifold X . Let Z C X be a closed subset. 

Let Z = lL=o be a Nash G-invariant stratification of Z . Let x be a character of G. Suppose 
that for any k e Z> and < i < I we have S*(Z h Sym k (CN^)) G ' x = 0. Then S X (Z) G ' X = 0. 

This proposition immediately follows from Corollary B.2.6 in [AGS . 
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Proposition 2.2.7. Let Gi be Nash groups acting on Nash manifolds Xi for i = l...n. Let 

Ei — > Xi be Gi-equivariant Nash vector bundles. 

(i) Suppose that S*(X j ,E j ) G i = for some 1 < j < n. Then 

n 

sr^Xt^Eip- * = o, 

i=l 

where Kl denotes the external product of vector bundles. 

(ii) Let Hi < Gi be Nash subgroups. Suppose that S*(Xi, Ei) Hi = S* (Xi, Ei) Gi for all i. Then 

^(IJjQ.&Eyn^ =s*(\Jx i ,ME i )^ G *, 

The proof is trivial and the same as the proof of Proposition 3.1.5 in |AGSj . 

Theorem 2.2.8 (Frobenius reciprocity). Let a unimodular Nash group G act transitively on a 
Nash manifold Z . Let ip : X Z be a G-equivariant Nash map. Let z G Z . Suppose that its 
stabilizer G z is unimodular. Let X z be the fiber of z. Let x be a character of G. Then S*(X) G,X 
is canonically isomorphic to S* (X z ) Gz,x . 

Moreover, for any G-equivariant bundle E on X, the space S*(X,E) G,X is canonically isomor- 
phic to S*(X Z ,E\ X J G " X . 

For proof see [XG2] . Theorem 2.3.8. 

2.2.4. Fourier transform and Homogeneity Theorem. 

From now till the end of the paper we fix an additive character k of F given by k(x) :— e 27riRc O)_ 

Notation 2.2.9. Let V be a vector space over F. Let B be a non- degenerate bilinear form on V . 
Then V defines Fourier transform with respect to the self-dual Haar measure on V . We denote it 
byT B :S*{V)^S*{V). 

For any Nash manifold M we also denote by Tb ■ S*(M X V) — > S*(M x V) the partial Fourier 
transform. 

If there is no ambiguity, we will write Ty instead Tb ■ 

Notation 2.2.10. Let V be a vector space over F. Consider the homothety action of F x on V 
by p(X)v :— \~ 1 v. It gives rise to an action p of F x on S*(V). 
Also, for any A £ F we denote ||A|| := \\\ dm ' F . 

Notation 2.2.11. Let V be a vector space over F. Let B be a non- degenerate symmetric bilinear 
form on V . We denote 

Z(B) := {x E V{F)\B(x,x) = 0}. 

Theorem 2.2.12 (Homogeneity Theorem). Let V be a vector space over F. Let B be a non- 
degenerate symmetric bilinear form on V . Let M be a Nash manifold. Let L C Sy^ xM (Z(B) xM) 
be a non-zero subspace such that V£ € L we have Tb(^) S L and B^ £ L (here B is interpreted as 
a quadratic form). 

Then there exist a non- zero distribution and a unitary character u of F x such that either 

p(A)£ = ||A||^w(A)£ for any A £ F x or p(A)£ = ||A||^ L+1 M (A)C for any A € F x . 

For proof see |AG2| . Theorem 5.1.7. 

We will also use the following trivial observation. 

Lemma 2.2.13. Let V be a finite dimensional vector space over F. Let a Nash group G act 
linearly on V . Let B be a G-invariant non- degenerate symmetric bilinear form on V . Let M be a 
Nash manifold with an action of G. Let £ £ S*(V(F) x M) be a G-invariant distribution. Then 
J~b(Q is olso G-invariant. 
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2.2.5. Harish- Chandra descent. 

Definition 2.2.14. Let an algebraic group G act on an algebraic variety X. We say that an 
element x S X(F) is G-semisimple if its orbit G{F)x is closed. 

Theorem 2.2.15 (Generalized Harish-Chandra descent). Let a reductive group G act on smooth 
affine varieties X and Y. Let \ be a character of G(F). Suppose that for any G-semisimple 
x G X(F ) we have 

S*((N^ x xY)(F)f^-x = 0. 
Then S*(X(F) x Y(F)) G ^^ = 0. 

For proof see [SSS], Theorem 3.1.6. 
2.3. D-modules and singular support. 

In this paper we will use the algebraic theory of D-modules. We will now summarize the facts that 
we need and give more details in Appendix [5] For a good introduction to the algebraic theory of 
D-modules we refer the reader to }Berj and [Borj . 

More specifically, we will use the notion of singular support of a distribution. For those who are 
not familiar with the theory of D-modules, Corollary 1 2 . 3 . 71 and the facts that are listed after it are 
the only properties of singular support that we use. 

In this subsection F = M. 

Definition 2.3.1. Let X be a smooth algebraic variety. Let £ € 5*(X(M)). Consider the Dj- 
submodule M.^ of S*(X(M.)) generated by £. We define the singular support of £ to be the 
singular support of M.^. We denote it by SS{^). 

Remark 2.3.2. A similar, but not equivalent notion is sometimes called in the literature a 'wave 
front of £ '. 

Notation 2.3.3. Let (V, B) be a quadratic space. Let X be a smooth algebraic variety. Consider 
B as a map B : V ~> V*. Identify T*(X x V) with T*X xVxV*. We define F v : T*(X x V) -> 
T*(X x V) by F v (a,v,(j)) := (a, 

Definition 2.3.4. Let M be a smooth algebraic variety and u be a symplectic form on it. Let 
Z C M be an algebraic subvariety. We call it M-coisotropic if one of the following equivalent 
conditions holds. 

(i) The ideal sheaf of regular functions that vanish on Z is closed under Poisson bracket. 

(ii) At every smooth point z € Z we have T Z Z D (T Z Z) . Here, (TzZ) 1 - denotes the orthogonal 
complement to {T Z Z) in (T Z M) with respect to u>. 

(Hi) For a generic smooth point z G Z we have T Z Z D (TzZ)- 1 . 
If there is no ambiguity, we will call Z a coisotropic variety. 

Note that every non-empty M-coisotropic variety is of dimension at least ^dimM. 

Notation 2.3.5. For a smooth algebraic variety X we always consider the standard symplectic 
form on T* X . Also, we denote by px ■ T* X — > X the standard projection. 

The following theorem is crucial in this paper. 

Theorem 2.3.6. Let X be a smooth algebraic variety. Let M. be a finitely generated Dx-module. 
Then SS(A4) is a T* X -coisotropic variety. 

This is a special case of Theorem I in [Gabj . For similar versions see also |KKS[ IMalj . 

Corollary 2.3.7. Let X be a smooth algebraic variety. Let £ e S*(X(R)). Then SS(£) is 
coisotropic. 
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We will also use the following well-known facts from the theory of D-modules. Let X be a 
smooth algebraic variety. 

Fact 2.3.8. Let £ G S*(X(R)). Then Supp(£) Zar = p x (SS(£))(R), where Supp(|) Xor denotes the 
Zariski closure o/Supp(£). 

Fact 2.3.9. 

Let an algebraic group G act on X. Let g denote the Lie algebra of G. Let £ G 5*(X(R)) G '( R - ) . 
Then 

SS{i) G {(x, 0) G T*X | Va G a <P{a(x)) = 0}- 

Fact 2.3.10. Let (V, B) be a quadratic space. Let Z C X xV be a closed subvariety, invariant with 
respect to homotheties in V. Suppose that Supp(£) G Z(R). Then SS(J-v(0) C Fy(p~^}^ v (Z)). 

For proofs of those facts see Appendix [Bj 
2.4. Specific notation. 

The following notations will be used in the body of the paper. 

• Let V := V n be the standard n-dimensional linear space defined over F. 

• Let sl(V) denote the Lie algebra of operators with zero trace. 

• Denote X := X n := sl(V„) x V n x V* 
. G := G n := GL(V4) 

• 0J= Sn ■= Lie(G„) = gl(K) 

• G := G n := G n x {1, er}, where the action of the 2-element group {1, a} on G is given by 
the involution g g l 

• We define a character x of G by y(G) = {1} and x(G — G) = {-1}. 

• Let G n act on G n +i, n +i and on sl(V n ) by := gAg^ 1 . 

• Let G act on V x V^* by g(v, 0) := (gv,g*<f>). This gives rise to an action of G on X. 

• Extend the actions of G to actions of G by cr(A) :— A f and a(v, 0) := (0*, v f ). 

• We consider the standard scalar products on sl(V) and V x V*. They give rise to a scalar 
product on X. 

• We identify the cotangent bundle T*X with X x X using the above scalar product. 

• Let Af := Af n C sl(V^) denote the cone of nilpotent operators. 

• G := (V x 0) U (0 x V*) G V x V* . 

• C : = x x V x 0) U (0 x V* x x V*) <z V x V* x V x V* . 

• Cxxx ■■= (sl(V) x V x x sl(V) x V x 0) U (sl(V) x x V* x s\(V) x x V*) G X x X. 

• S := {(A, v, 0) G X n \A n = and 4>(A l v) = for any < i < n}. 

5 := {((^x, 2 ), (4 2) «2, 2 )) G X x X | Vi, j G {1,2} 

{Ai , vj , <j>j ) G 5 and Va G g\(V),g(Ax,Vx,^i)L(A i ,v 2 ,<l>2)} 

• Note that 

S = {((At, vi, <h), (A 2 ,v 2 , 0a)) G X x X | Vi, j G {1, 2} 

(Ai,Vj,<j>j) G 5 and [Ax, A 2 ] + vi ® 02 - "2 <8> 0i = 0}. 

• 5" := S 1 — Gxxx- 

• T := {(v, <f)) <EV xV*\ (j)(V) = 0}. 

• For any AeFwe define v\ : X -> X by z/ A (A i>, 0) := {A + Xv <g) - A-^-Id, v, 0). 
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• It defines v\ : X x X — > X x X. It is given by 

((Ai,vx,(j> 2 ), (A 2> v 2 ,<h)) = 

= ((A x + A«i »<f>i- X ^ ld, vi,<j>!), (A 2 ,v 2 - XA 2Vl ,4> 2 - XA*^)). 

n 

3. Harish-Chandra descent 

3.1. Linearization. 

In this subsection we reduce Theorem A to the following one 
Theorem 3.1.1. S*(X(F)) S >* = 0. 

We will divide this reduction to several propositions. 

Proposition 3.1.2. If V(G n+1 (F)) G "^ = then Theorem A holds. 

The proof is straightforward. 
Proposition 3.1.3. If S* (G„+i(F)) 5 »>x = then V(G n+1 {F)) G -* = 0. 

Follows from Theorem 12.2.51 

Proposition 3.1.4. J/<S*(0 n+1 (F)) 5 «( F )'X = then S*(G n+ i(F)) G ^' x = 0. 

Proof. Let £ G S*(G n+1 (F)) G '^ F ^ x . We have to prove £ = 0. Assume the contrary. Take 
p E Supp(£). Let t = det(p). Let / G S(F) be such that / vanishes in a neighborhood 
of and f(t) 0. Consider the determinant map det : G n +i{F) — > F. Consider £' := 
(/ o det) • f. It is easy to check that f G S*(G„+i(.F))°»( F )'X and p G Supp(f)- However, 
we can extend £' by zero to f" G S*(g„ + i(F)) G "( F )'X, which is zero by the assumption. 
Hence £' is also zero. Contradiction. □ 

Proposition 3.1.5. If S*(X n (F)) S ^ F ^ = tfien 5*(g„ + i(F)) 5 -( F )^ = o. 

Proof. The G n (F)-space gl n+1 (F) is isomorphic to X n (F) x F x F with trivial action on 
F x F. This isomorphism is given by 

A ^ n \ xl )»((A-—1d,v,<l>),\,TrA). 

01xn A y ?1 

□ 

3.2. Harish-Chandra descent. 

Now we start to prove Theorem 13.1.11 The proof is by induction on n. Till the end of 
the paper we will assume that Theorem 13.1.11 holds for all k < n for both archimedean 
local fields. 

The theorem obviously holds for n = 0. Thus from now on we assume n > 1. The goal 
of this subsection is to prove the following theorem. 

Theorem 3.2.1. S*(X(F) - S(F)) G ^ = 0. 

In fact, one can prove this theorem directly using Theorem 12.2.151 However, this will 
require long computations. Thus, we will divide the proof to several steps and use some 
tricks to avoid part of those computations. 
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Proposition 3.2.2. S*(X(F) -(jVxFx V*)(F)) G ^ = 0. 

Proof. By Theorem 12.2.151 it is enough to prove that for any semi-simple A £ sl(V) we 
have 

S*((N s W A x(VxV*))(F)f A >* = 0. 

Now note that Ga(F) = Y\G ni (Fi) where rii < n and Fj are some field extensions of 
F. Note also that 

( n ga,a V*){F) = &(V)a x (V x V*)(F) = Y[X ni (Fi) x 2(sl(^) A )(F), 

where ^(s^V)^) is the center of s1(V)a- Clearly, Ga acts trivially on Z(s\(V)a). 
Now by Proposition 12.2.71 the induction hypothesis implies that 

S*(l[x nt (F t ) x Z{BL(V)A){F)) u8n *W' x = 0. 

□ 

In the same way we obtain the following proposition. 
Proposition 3.2.3. S*(X{F) - (sl(V) x T)(F)) d ^ = 0. 
Corollary 3.2.4. S*(X(F) - (TV x r)(F)) 5 >* = q. 

Lemma 3.2.5. Let A s sl(V), jj £ V and <f> £ V* . Suppose A + Xv ® is nilpotent for all 
Xe F. Then ^(A^) = /or any i > 0. 

Proof. Since A + Xv (g) <f> is nilpotent, we have tr(A + Xv (£> <fi) k = for any k > and A G F. 
By induction on z this implies that ^(A'w) =0. □ 

Proof of Theorem 1 3. 2. 1\ By the previous lemma, {~]\ e pV\(M x ^) C S. Hence 

U AeF ^-^xr)Di-5. 

By Corollary 5* - (TV x r)(F)) G 'X = 0. Note that v x commutes with the 
action of G. Thus S*{v x {X{F) - (TV x r)(F))) 5 >* = q an d hence <S*(X(F) - S(F)) d ^ = 
0. □ 

4. Reduction to the geometric statement 

In this section coisotropic variety means X x X-coisotropic variety. 

The goal of this section is to reduce Theorem l3.1.1l to the following geometric statement. 

Theorem 4.0.1 (geometric statement). For any coisotropic subvariety ofTcS we have 
T C Cxxx- 

Till the end of this section we will assume the geometric statement. 

Proposition 4.0.2. Let £ e S*(X(F)) d >* = 0. Then Supp(£) C (sl(V) x C)(F). 

Proof for the case F = R. 

Step 1. SS(C) C S. 
We know that 

Supp(e),Supp(^ s - 1 y) 0,Supp(^-i l/ ,(0),Supp(^ 1 (e)) C S(F). 
By Fact 12.3.101 this implies that 

SS(C) C (Sx X) n F sKV) {S x X) D F VxV *{S x X) D F X (S x X). 
On the other hand, £ is G-invariant and hence by Fact 12.3.91 

SS(C) C {{(x u x 2 ) eXxX\Vge 0, g{ Xl )±x 2 }. 
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Thus C S. 

Step 2. SS(0 C C X y.x- 
By Corollary 12.3.61 1S X x X-coisotropic and hence by the geometric statement 

SS(£) c C M . 

Step 3. Supp(0 C (sl(V) x C)(F). 
Follows from the previous step by Fact 12.3.81 □ 

The case F = C is proven in the same way using the following corollary of the geometric 
statement. 

Proposition 4.0.3. Any (X x X)c~coisotropic subvariety of Sc is contained in (Cxxx)c- 

Now it is left to prove the following proposition. 
Proposition 4.0.4. Let £ e S*(X(F )) G ( F )>x be such that 

Supp(£), Supply, (£)) C (81(10 x C)(F). 

Then £ = 0. 

4.1. Proof of proposition I4.07H 

Proposition 14.0.41 follows from the following lemma. 

Lemma 4.1.1. Let F x act onV x V* by X(v, cj)) := [Xv, f ). Let £ e S*((V x V*)(F)) F>< 
be such that 

Supp(0,SupppVxv(0) C C ( F )- 

Then £ = 0. 

By Homogeneity Theorem (Theorem 12.2. 12j) it is enough to prove the following lemma. 

Lemma 4.1.2. Let n be a character of F x given by || • \\ n u or || • where u is 

some unitary character. Let F x x F x act on V x V* by (x,y)(v,(j>) = (-v, —(f)). Then 

S* {VxV , )(F) (C(F)r xFX -^=0. 

By Proposition 12.2.61 this lemma follows from the following one. 
Lemma 4.1.3. For any k > we have 

ft) S*(((V - 0) x 0)(F),Sym k (CN^ v x J )x0 (F))) FXxFX ^ = 0. 
(ii) S*((0 x (V* 0))(F),SymHCN^:_ Q) (F))r x xFX >^ = 0. 
(Hi) S*{0,Sym k {CN^ xV * (F))) FX xpx '^ xl = 0. 

Proof. 

(i) Cover V — by standard affine open sets := {xi ^ 0}. It is enough to show that 
5*((y i x0)(F) ! %m fc (C^j; )(F) (F))) FX ^ x ^>< 1 = 0. 

Note that V* is isomorphic as an F x x F x -manifold to F n ~ 1 x F x with the action given 
by (x, y) (v, a) = (v, ^a). Note also that the bundle Sym k (CN^/ < x0 ^ F ^(F)) is a constant 
bundle with fiber Sym k {V). 

Hence by Proposition ^TH\ it is enough to show that S*(F X , Sym k (V)) F>< xfX '' lXl = 0. 
Let H := (F x x F x ) 1 = {(t, t) e F x x F x }. Now by Frobenius reciprocity ( theorem |2~2T5|) 
it is enough to show that (Sym k (V*(F)) ®r C) ff,M><1 l ff = 0. This is clear since (t,t) acts 
on (Sym k (V*(F)) by multiplication by t~ 2k . 

(ii) is proven in the same way. 

(iii) is equivalent to the statement ((Sym k (V x V*)(F)) ® R C) F>< xF * <^ xl = 0. This is 
clear since (£, 1) acts on Sym k (V x V*)(jF) by multiplication by t~ k . □ 
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5. Proof of the geometric statement 
5.1. Preliminaries on coisotropic subvarieties. 

Proposition 5.1.1. Let M be a smooth algebraic variety with a symplectic form on it. Let 
R C M be an algebraic subvariety. Then there exists a maximal M- coisotropic subvariety 
of R i.e. an M-coisotropic subvariety T C M that includes all M -coisotropic subvarieties 
ofR. 

Proof. Let T 1 be the union of all smooth M-coisotropic subvarieties of R. Let T be the 
Zariski closure of T' in R. Clearly, T includes all M-coisotropic subvarieties of R. Let U 
denote the set of regular points of T. Clearly U D T' is dense in U. On the other hand, for 
any i£t/n T', the tangent space to T at x is coisotropic. Hence T is coisotropic. □ 

Remark 5.1.2. Suppose M is affine. Then T can be computed explicitly in the following 
way. Let I be the ideal of regular functions that vanish on R. We can iteratively close it 
with respect to Poisson brackets and taking radical. Since O(M) is Noetherian, this process 
will stabilize. Let J denote the obtained closure and Z(J) denote the zero set of J. Then 
T = Z(J)C\R. 

The following lemma is trivial. 

Lemma 5.1.3. Let M be a smooth algebraic variety and to be a symplectic form on it. Let 
a group G act on M preserving u>. Let S be a G -invariant subvariety. Then the maximal 
M-coisotropic subvariety of S is also G-invariant. 

Definition 5.1.4. Let Y be a smooth algebraic variety. Let Z <ZY be a smooth subvariety 
and R C T*Y be any subvariety. We define the restriction R\z C T*Z of R to Z in 
the following way. Let R' = p Y l {Z) R R. Let q : p Y 1 (Z) — ► T* Z be the projection. We 
define R\z '■— q(R'). 

Lemma 5.1.5. Let Y be a smooth algebraic variety. Let Z C Y be a smooth subvariety 
and R C T*Y be a coisotropic subvariety. Assume that any smooth point z G p Y (Z) D R 
is also a smooth point of R and we have T z (p Y 1 (Z) n R) — T z (p Y (Z)) n T Z R. 
Then R\z is T* Z coisotropic. 

In the proof we will use the following straightforward lemma. 

Lemma 5.1.6. Let W be a linear space. Let L c W be a linear subspace and R C W ® W* 
be a coisotropic subspace. Then R\l is L © L* coisotropic. 

Proof of lemma \5.1.5[ Without loss of generality we assume that R is irreducible. Let 
R' = p Y 1 (Z) n R. Without loss of generality we assume that R' is irreducible. Let R" 
be the set of smooth points of R 1 . Let q : p Y x (Z) — > T* Z be the projection. Let R'" be 
the set of smooth points in q(R"). Clearly R'" is dense in R\z- Hence it is enough to 
prove that for any x e R'" the space T x (R\z) is coisotropic. Let y e R" s.t. x — q(y). 
Denote W := T Py{v) Y. Let Q := T y R cW®W*. Let L :— T Py[v) Z. By the assumption 
T x (R\z) D Q\l- By the lemma, Q\l is coisotropic and hence T x (R\z) is also coisotropic. □ 

Corollary 5.1.7. Let Y be a smooth algebraic variety. Let an algebraic group H act on 
Y . Let q : Y — > B be an H-equivariant morphism. Let O C B be an orbit. Consider the 
natural action of G on T*Y and let R C T*Y be an H-invariant subvariety. Suppose that 
p Y (R) C q^iO). Let x e O. Denote Y x := q^ 1 (x). Then 
— if R is T*Y -coisotropic then R\y x is T*(Y X ) -coisotropic. 
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Corollary 5.1.8. In the notation of the previous corollary, if R\y x has no (non-empty) 
T*(Y X )- coisotropic subvarieties then R has no (non-empty) T*(Y)- coisotropic subvarieties. 

Note that the converse statement does not hold in general. 
5.2. Reduction to the Key Proposition. 

In this subsection coisotropic variety means X x X-coisotropic variety. 
We will use the following notation. 

Notation 5.2.1. 

(i) For any nilpotent operator A G s\(V) we denote 

Q A := {(v, <j)) e V x V* | v ® 4> G [A, g}} = {(«, q>)eVxV* \ (v ® 4>)±q a }- 

(ii) Denote by T the maximal coisotropic subvariety of S' . 
(Hi) For any two nilpotent orbits 0\, 2 C N denote 

U(Ox,0 2 ) := {{A 1 ,v 1 ,<l> l ,A 2 ,V2,h) e X x X|Vt,j G {1,2} 

Ai g Oi, (vj,<f)j) G Q Al , [Ai,A 2 ] + vi ® 02 - v 2 ® fa = 0, (v^fa^^fa) £ C}. 

The geometric statement is equivalent to the following theorem 

Theorem 5.2.2. T = 0. 

The goal of this subsection is to reduce the geometric statement to the following Key 
Proposition. 

Proposition 5.2.3 (Key Proposition). For any two nilpotent orbits 0\,0 2 there are no 
(non-empty) coisotropic subvarieties in U(0\,0 2 ). 

The reduction will be in the spirit of the beginning of section 3 in [AG RS . 

Notation 5.2.4. Let 

M i = {(A-i,A 2 ) GAAxAA|dimG(A 1 )+dimG( J 4 2 ) < i}. 

Af* := {{A^fa^A^fa) G S'\(A U A 2 ) gTV 4 }. 

We will prove by descending induction that T C M l . From now on we fix i, suppose that 
this holds for i and prove that holds for i — 1. Let 6 denote the subgroup of automorphisms 
of X x X generated by F sl (v) and Fyxv- 

Denote M l := n„ GS v {N l ). We know that T C TV 1 , and hence T C Af i . Let U' 1 := 
j\fi _j\fi-i _ it i s enough to show that U l does not have (non-empty) coisotropic subvarieties. 

Notation 5.2.5. Let Oi,0 2 be nilpotent orbits such that dimOi + dim02 = i- Denote 
U'(0 1 ,0 2 ) := {(A 1 ,v 1 ,fa,A 2 ,v 2 , fa) G t/ l |Ai 6 Ox, A 2 g 2 }- 

Since the sets U'(Oi,0 2 ) form an open cover of U l , it is enough to show that each 
U'(Oi,0 2 ) does not have (non-empty) coisotropic subvarieties. This fact clearly follows 
from the Key Proposition using the following easy lemma. 

Lemma 5.2.6. U'{O u 2 ) C U(O x , 2 ). 



AVRAHAM AIZENBUD AND DMITRY GOUREVITCH 



5.3. Reduction to the Key Lemma. 

We will use the following notation 

Notation 5.3.1. 

:= {(vi,<f>t,v2,<fa) e Qax Qa- C\ 

3B G [A, g] n TV such that [A, B] + vi ® 4> 2 ~ v 2 <8 <f>i = 0}. 

The goal of this subsection is to reduce the Key Proposition to the following Key Lemma. 

Lemma 5.3.2 (Key Lemma). Ra does not have (non-empty) V x V* x V x V* -coisotropic 
subvarieties. 

Notation 5.3.3. Denote 

U"(O u 2 ) '•= {(Ai,vi,<pi,A 2 ,V2,<f>2) e U(0 1 ,0 2 )\ba 1 -Lqa 2 }- 

Lemma 5.3.4. Any X x X-coisotropic subvariety of U(Oi,0 2 ) lies in U"(Oi,0 2 ). 

Proof. Denote M := O x XV X V* X 2 X Vx V*. Note that U{0 1 ,0 2 ) C M. Note that any 
coisotropic subvariety of M is contained in M' := {(Ai, v\, <j)\, A 2 , V2, (fi 2 ) 6 M \ qa 1 -L9a 2 }- 
Hence any coisotropic subvariety of U(Oi 7 2 ) is contained in U(Oi,0 2 ) H M'. □ 

The following straightforward lemma together with Corollarv l5. 1.81 finish the reduction. 

Lemma 5.3.5. U"(Oi,0 2 )\axVxv C R A - 

5.4. Proof of the Key Lemma. 

We will first give a short description of the proof for the case when A is one Jordan block. 
Then we will present the proof in the general case. 

During the whole subsection coisotropic variety means V x V* x V x V^*-coisotropic 
variety. 

5.4.1. Proof in the case when A is one Jordan block. 
In this case Q A = \S=o( KerAl ) x {Ker(A*) n ~ l ). Hence 

n 

QaxQ a = (J (KerA 1 ) x (Ker(A*) n -' 1 ) x (KerA j ) x (Ker{A*) n - j ). 

Denote L t] := (KerA 1 ) x ( J ftr e r(A*)"- 4 ) x (KerA^) x (i^er(A*) n ^'). 

It is easy to see that any coisotropic subvariety of Qa x Qa is contained in 1J" =0 ^U- 
Hence it is enough to show that for any i, dimi?^ n L« < In. For i = 0, n it is clear since 
Ra H Lji is empty. So we will assume < i < n. 

Let / e O(Lii) be the polynomial defined by f(v\, 4>i, v 2 , fa) ■= (vi)i(<fa)i+i - 
(v 2 )i((j)x)i+i, where (-)j means the i-th coordinate. It is enough to show that f(RA n£,:i) = 
{0}. ' 

Let (f>i,V2, ^2) S Let M := ui ® <f>2 — ^2 ® <^i. Clearly, M is of the form 
M = ( ° 2Xl * ) . 

\ 0(n— i)xi 0(n— i) X (ri— i) J 

Note also that M iyi+ i — f(vi,(f>i,V2,<f> 2 )- 

It is easy to see that any B satisfying [A, B] = M is upper triangular. On the other 
hand, we know that there exists a nilpotent B satisfying [A, B] = M. Hence this B is 
upper nilpotent, which implies M^+i = and hence f(vi,4>i,v 2 ,4>2) = 0. 
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5.4.2. Notation on /titrations. 
Notation 5.4.1. 

(i) Let L be a vector space with a gradation G l L. It defines a filtration G- l L by 

(•■!■■■ e, • 

(ii) Let L be a vector space with a descending filtration F- 1 . We define F >1 L := 

U, ,/ '/•• 

Notation 5.4.2. Let L and M be vector spaces with descending filtrations F- l L and 

Define filtrations F^ l (L ® M) := £ fc+;=l F ~ kL ® F~ l M and F^ l (L*) := (F > - i L) ± . 
Similarly for gradations G l L and G l M we define gradations G l (L © M) := 
fc+i=l G k L ® G l M and G i (L*) := (©^ &L)±. 

We fix a standard basis {E, H, F} of SI2. 

Notation 5.4.3. Let L be a representation 0/SI2. We define 

- A gradation W a {L) := Ker(H — aid) and 

- An ascending filtration K{(L) := Ker(E l ). 

Note that if L is an irreducible representation then Ki(L) — W- d%mL+1 ~ 2t {L). 

5.4.3. Proof of the Key Lemma. 

We will cover Ra by linear spaces and show that every one of them does not include 
coisotropic subvarieties of Ra- 

Fix a morphism of Lie algebras ip : SI2 — > sl(V) such that ip(E) = A. Decompose V to 
irreducible representations of SI2: V = © i=1 V% such that dimVi > dim K+i. 

Notation 5.4.4. Denote Di := dimV^. Let D denote the multiindex D := (D\, ...,D k ). 
For any multiindex I = (I\, I k ) such that < J; < Di, 1^0 and I ^= D we define 

- Li ■.= W^ +1 - 2h (V 1 )(B...(BW^ D * +1 - 2I »(V k ) = K^Vi) ® ... ® K Ilt (V k ) 

- L'j :=W^ Dl+1 - 2Il (V 1 *)®...®W^ Dk+1 - 2Ik (V k *)=K Il (V 1 *)(B...®K Ik (V^) 

- Lu := Li x L' D _ I x Lj x L' d _j 

The following two lemmas are straightforward 

Lemma 5.4.5. 

Ra c (J Lu 

i,j 

Lemma 5.4.6. Lu is not coisotropic if I ^ J. 

Hence it is enough to prove the following proposition. 

Proposition 5.4.7. dimL// f) Ra < 2n. 

From here on we fix / and suppose that the proposition does not hold for this I. Our 
aim now is to get a contradiction. Note that if Proposition 15.4.71 holds for / then it holds 
for D — I. Hence without loss of generality we can (and will) assume I k < D k . 

Lemma 5.4.8. For any m < I we have D m — Di > I m — Ii > 0. 

Before we prove this lemma we introduce some notation. 
We fix a Jordan basis of A in each Vi. 

Notation 5.4.9. For any v S V, <fi € V* 7 X G V ® V* we define v l to be the l-th component 
of v with respect to the decomposition V = ®Vi and v a to be its a coordinate. 
Similarly we define 4> l , <p a , X lm , X 1 ™^ 
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Proof of lemma \5.4-8\ It is enough to prove that for any I, m we have + (D m — I m ) < 
max{Di, D m ). Assume that the contrary holds for some I, m. It is enough to show that in 
this case dim(Q J 4 n (Lj x L' D _j)) < n. Consider the function g G 0(Lj x L' D _ I ) defined 
by g(v, <f>) = 4>T m +i ' w /( • ^ i s enough to show that g(QA H (Li x Lrj-i)) = {0}. 

Let fi £ K,® VJ* be defined by B a> p = S a -pj m -i l+ i. Consider B as an element 
of g. Note that B G qa and (B,v 0) = g(v,<j>) for any (w, 0) e Lj x L' D _ I . Hence 
<?(Q A n {Li x L D _ Z )) = {0}. □ 

Corollary 5.4.10. 

(i) If I m = then J; = /or any I > m. 

(ii) If I m = D rn then Ii = Di for any I > m. 

Corollary 5.4.11. D x > h > 0. 

Notation 5.4.12. Let k' be the maximal index such that D^i > I^ > 0. 
Notation 5.4.13. Define ft G 0(V x V* x V x V*) by 

fi(vi, fa,V2, 2 ) := (^(^Ui-^/.O^+i- 
De/ine also f -^Li^ fl- 

Now it is enough to prove the following proposition. 
Proposition 5.4.14. 

/(iU n l„) = {o}. 

We will need several notations and straightforward lemmas. 
Lemma 5.4.15. For any a < \D m -D{\ we have W^ a (Vi <8> V£) = {X G Vj <8> V£\E(X) G 

Definition 5.4.16. £>e/me gradation Wj on Vi by W}(Vi) = W l+ ^ Dl+l - 2Il \Vi). It gives 
rise to gradations W} on V t * , V m ® V* , V, V* . 

Lemma 5.4.17. 

(i) If i is odd then Wj = 0. 

(ii) W}\V) = Li. 
(tit) Wp(V*) = L' D _j. 

Definition 5.4.18. Let A be the algebra Wf°(V®V*) andl be its ideal Wf~°(V®V*) = 
Wf{V®V*). Clearly A/1 = \[End(W}{V)). This gives rise to a homomorphism 
e : A — > End(Wf(V)). 

Lemma 5.4.19. 

ft) A = 0i< ; , m <, w^'-^-^-^m v; n ). 

(ii) 1 := e i<; , m<fc W >A-u»-2(/,-7 m ) +2(V j ® v-). 
(m,) dim(W?(V)) = fc' 

(iv) Consider the basis on Wf(V) corresponding to the one on V and identify End(Wj(V)) 
with gl(fc'). Then 

Corollary 5.4.20. A = {X E End(V)\[A,X] G X}. 

Proof. Follows from the previous lemma using Lemma 15.4.151 □ 
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Proof of Proposition \5 .^.Tl\ Let (vi, 4>x,Vi, ^2) € Ljj n Let M := i>i <8> 02 — ^ 2 <8> <^i • 
We know that there exists a nilpotent matrix £? £ [j4, gl(V)] such that [A, £?] = M. By 
Corollary [5X20]B G A Denote A := e(B). Fix 1 < I < k'. Denote a; := M\\ Jl+v Note 
that = M u . Hence = ... = B^ 1 ' = A„ = B{{ +1J,+1 - a t = ... = B° ,D ' - a h 

Since B e [A, End(V)] we have tr(B u ) = 0. Thus A« = ^^ty. 

Since B is nilpotent A is nilpotent. Hence tr(A) — and thus Yli=i ^d/' n ' = ^ which 
means /(ui, 01,^2, ^2) = 0. □ 

Appendix A. Theorem A implies Theorem B 

This appendix is analogous to section 1 in AGRSJ. There, the classical theory of Gelfand 
and Kazhdan (see [GK] ) is used. Here we use an archimedean analog of this theory which 
is described in [AGSj . section 2. We will also use the theory of nuclear Frechet spaces. For 
a good brief survey on this theory we refer the reader to |CHM) . Appendix A. 

Notation A. 0.1. 

(i) For a smooth Frechet representation ir of a real reductive group we denote by tt the 
smooth dual of it. 

(ii) For a representation 7r of GL n (F) we let if be the representation of GL n (F) defined by 
7? = 77 o 6, where 9 is the (Cartan) involution 9(g) = g^ 1 . 

We will use the following theorem. 

Theorem A. 0.2 (Casselman - Wallach globalization). Let G be a real reductive group. 
There is a canonical eguivalence of categories between the category of admissible smooth 
Frechet representations of G and the category of admissible (g,K)- modules. 

See e.g. |Wal| . chapter 11. 

We will also use the embedding theorem of Casselman. 

Theorem A. 0.3. Any irreducible (jj, K)-module can be imbedded into a (g, K) -module of 
principal series. 

Those two theorems have the following corollary. 

Corollary A. 0.4. The underlying topological vector space of any admissible smooth 
Frechet representation is a nuclear Frechet space. 

Definition A. 0.5. Let G and H be real reductive groups. Let (tt,E) and (r, W) be ad- 
missible smooth Frechet representations of G and H respectively. We define 77 ® r to be 
the natural representation of G x H on the space E(§W. 

The following proposition is well known. For the benefit of the reader we include its 
proof in subsection I A. 11 

Proposition A. 0.6. Let G and H be real reductive groups. Let 77 and r be irreducible 
admissible Harish- Chandra modules of G and H respectively. Then 77 ® r is irreducible 
Harish- Chandra module of G x H . 

Corollary A. 0.7. Let G and H be real reductive groups. Let 77 and r be irreducible ad- 
missible smooth Frechet representations of G and H respectively. Then 77 (g) r is irreducible 
representation G x H . 

Lemma A. 0.8. Let G and H be real reductive groups. Let (77, E) and (r, W) be admissible 
smooth Frechet representations of G and H respectively. Then Homc{ 7T , T ) is canonically 
embedded to H omc (77 <g) r, C). 
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Proof. For a nuclear Frechet space V we denote by V' its dual space equipped with the 
strong topology. Let W denote the underlying space of t. By the theory of nuclear Frechet 
spaces, we know Hom c (E,W) = E'®W and Hom c {E®W ,£) = E'®W . The lemma 
follows now from the fact that W is canonically embedded to W. □ 

We will use the following two archimedean analogs of theorems of Gelfand and Kazhdan. 

Theorem A. 0.9. Let n be an irreducible admissible representation of GL n (F). Then 7? 
is isomorphic to tt. 

For proof see [AGS] , Theorem 2.4.4. 

Theorem A. 0.10. Let H C G be real reductive groups and let a be an involutive 
anti- automorphism of G and assume that o~(H) = H . Suppose cr(£) = £ for all H-bi- 
invariant Schwartz distributions £ on G. Let 7r be an irreducible admissible smooth Frechet 
representation of G. Then 

dimHom^f (tt, C) • dimHom# (tt, C) < 1. 

For proof see [AGS] , Theorem 2.3.2. 

Corollary A. 0.11. Let H C G be real reductive groups and let a be an involutive anti- 
automorphism of G such that o~(H) = H. Suppose = £ for all Schwartz distributions 
£ on G which are invariant with respect to conjugation by H . 

Let 7r be an irreducible admissible smooth Frechet representation of G and t be an 
irreducible admissible smooth Frechet representation of H. Then 

dimHoiiiH (tv, t) • dimHom# (tt, t) < 1. 

Proof. Define a 1 : G x H -» G x H by a'(g, h) := (a(g),a(h)). Let AH < G x H be the 
diagonal. Consider the projection G x H — > H . By Frobenius reciprocity (Theorcm l2.2.8|) . 
the assumption implies that any Aif-bi-invariant distribution on G x H is invariant with 
respect to a'. 

Hence by the previous theorem, for any irreducible admissible smooth Frechet 
representation tt' of G x H we have dimHomAff(7r', C) • dimHoniAH (tt', C) < 1. 

Taking tt' := n ® f we obtain the required inequality. □ 

Corollary A. 0.12. Theorem A implies Theorem B. 

Proof. By Theorem IA.0. 91 dimHoniff (tt,t) = dim Horn// (tt, t) = dim Horn// (tt, t). □ 
A.l. Proof of proposition IA.0.61 

Notation A. 1.1. Let G be a reductive group, q be its Lie algebra and K be its maximal 
compact subgroup. Let n be an admissible (g, K)-module. 
Let p be an irreducible representation of K. 

(i) We denote by e p : n — > 7r the projection to the K-type p. 

(ii) We denote by G™ the subalgebra of End(e p (Tr)) generated by the actions of K and 
e p U(Q)e p . 

The following lemma is well-known 

Lemma A. 1.2. Let tt be an irreducible admissible (g, K)-module. Let p be an irreducible 
representation of K . Suppose that e p (7r) ^ 0. Then e p (ii) is an irreducible representation 
ofG*. 
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We will also use Bernside theorem. 

Theorem A. 1.3. Let V be a finite dimensional complex vector space. Let A C End(V) 
be a subalgebra such that V does not have any non-trivial A-invariant subspaces. Then 
A = End(V). 

Now we are ready to prove proposition I A. ."HI 

Proof of vrovosition \A~.0.6l Let g and f) be the Lie algebras of G and H. Let K and L 
be maximal compact subgroups of G and H. Let u> C ir <S> r be a nonzero (g x ty,K x 
L)-submodule. Then u> intersects non-trivially some K x L type. Denote this type by 
p ® a. By Lemma IA.1.21 e p (7r) is an irreducible representation of G p and e ff (r) is an 
irreducible representation of H^. Hence by Bernside theorem G n p — End(e p (Tr)) and H£ = 
End{e a {T)). Hence (GxiJ)^ = End(e p (ir)®e a (T)). Thus wne p(8(T (7r®-r) = e^^r). 
This means that u> contains an element of the form v <S> w, which implies that lo — 

7T (g) T. □ 



Appendix B. D-modules 

In this appendix X denotes a smooth affine variety defined over K.. All the statements 
of this section extend automatically to general smooth algebraic varieties defined over R. 
In this paper we use only the case when X is an affine space. 

Definition B.0.1. Let D(X) denote the algebra of polynomial differential operators on X . 
We consider the filtration F- l D(X) on D(X) given by the order of differential operator. 

Definition B.0.2. We denote by Gr D(X) the associated graded algebra of D{X). 

Define the symbol map a : D{X) — > Gr D(X) in the following way. Let d G D(X). 
Let i be the minimal index such that d G F- 1 . We define <r(d) to be the image of d in 
(F^DiXjj/iF^DiX)) 

Proposition B.0.3. Gr D(X) = 0(T*X). 

For proof see e.g. [Borj . 

Notation B.0.4. Let {V, B) be a quadratic space. 

(i) We define a morphism of algebras $y : D(X x V) — > D(X x V) in the following way. 
Consider B as a map B : V — » V* . For any f G V* we set < I > y(/) := dg-itfy For any 

v G V we set $>y(d v ) := -B(v) and for any d G D(X) we set §y{d) := d. 

(ii) It defines a morphism of algebras $y : 0(T*X) — > 0(T*X). 

The following lemma is straightforward. 
Lemma B.0.5. Let f be a homogeneous polynomial. Consider it as a differential operator. 

The D-modules we use in the paper are right D-modules. The difference between right 
and left D-modules is not essential (see e.g. section VI. 3 in |Bor] ) . We will use the notion 
of good filtration on a D-module, see e.g. section II. 4 in [Borj . Let us now remind the 
definition of singular support of a module and a distribution. 

Notation B.0.6. Let M be a D{X)-module. Let a G M be an element. Then we denote 
by Ann D ( X ) the annihilator of a. 
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Definition B.0.7. Let M be a D(X)-module. Choose a good filtration on M. Consider 
grM as a module over GrD(X) = 0(T*X). We define 

SS(M) := Supp(GrM) C T*X. 

This does not depend on the choice of the good filtration on M (see e.g. jBorj . section 

114). 

For a distribution £ £ S*(X(M)) we define SS(£) to be the singular support of the module 
of distributions generated by £. 

The following proposition is trivial. 

Proposition B.0.8. Let L < D(X) be a right ideal. Consider the induced filtrations on L 
andD(X)/I. Then Gr(D(X)/I) Gr(Z)pf))/ Gr(I). 

Corollary B.0.9. Let £ e S*(X). Then SS(£) is the zero set of Gr (Ann D ( X) £). 

Corollary B.0.10. Let I < 0(T*X) be the ideal generated by {cr(d) \ d E Ann D ( X )(0} ■ 
Then SS(£) is the zero set of I. 

Corollary B.0.11. Fact\2JLE holds. 

Lemma B.0.12. Let £ 6 S*(X). Let Z C X be a closed subvariety such that Supp(£) C 
Z(R). Let f £ O(X) be a polynomial that vanishes on Z. Then there exists k £ N such 
that / fe £ = 0. 

Proof. 

Step 1. Proof for the case when X is affine space and / is a coordinate function. 
This follows from the proof of Corollary 5.5.4 in |AG1| . 

Step 2. Proof for the general case. 
Embed X into an affine space A N such that / will be a coordinate function and consider 
£ as distribution on A N supported in X. By Step 1, / fc £ = for some k. □ 

Corollary B.0.13. Fact\MM holds. 

Proposition B. 0.14. Fact \2.S.lO\ holds. Namely: 

Let (V, B) be a quadratic space. Let Z C X x V be a closed subvariety, invariant 
with respect to homotheties in V. Suppose that Supp(£) C Z(M). Then SS(J-\/(S,)) C 
F v { Px \ v {Z)). 

Proof. Let / <E 0(X x V) be homogeneous with respect to homotheties in V. Suppose 
that / vanishes on Z. Then $y(/ fe ) £ Ann D ( X )(3 7 v(0)- Therefore a(<by(f k )) vanishes 
on SS{T v {0)- On the other hand, a(^(f k )) = $0(cr(/ fc )) = ($0(cr(/))) fc . Hence 
SS^viO) i s included in the zero set of $y(cr(/)). Intersecting over all such / we obtain 
the required inclusion. □ 
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